Imaged image data processing apparatus, viewing information creating apparatus, viewing information creating system, imaged image data processing method and viewing information creating method

ABSTRACT

An imaged image data processing apparatus includes obtaining means for obtaining imaged image data, which is imaged by an imaging section that images the sight of a user, extracting means for extracting identification data for identifying a viewed program from the imaged image data obtained by the obtaining means, and transmitting means for transmitting the identification data extracted by the extracting means to an external viewing information creating apparatus.

CROSS REFERENCES TO RELATED APPLICATIONS

This application is a continuation application of and claims the benefitunder 35 U.S.C. §120 of U.S. patent application Ser. No. 14/024,207,titled “IMAGED IMAGE DATA PROCESSING APPARATUS, VIEWING INFORMATIONCREATING APPARATUS, VIEWING INFORMATION CREATING SYSTEM, IMAGED IMAGEDATA PROCESSING METHOD AND VIEWING INFORMATION CREATING METHOD,” filedSep. 11, 2013, which is a divisional application of Ser. No. 12/079,990,filed Mar. 31, 2008, which claims the benefit under 35 U.S.C. §119 ofJapanese Patent Application 2007-096353 filed in the Japanese PatentOffice on Apr. 2, 2007, each of which is hereby incorporated byreference in its entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an imaged image data processingapparatus and imaged image data processing method, which performprocessing on imaged image data of a subject sight, and viewinginformation creating apparatus, viewing information creating method andviewing information creating system, which create viewing information.

2. Description of the Related Art

In the past, the ratings for television broadcasting programs aremeasured by unique machines placed partial households selected at randomby a company providing a ratings calculation service and are averaged bya statistical method.

Systems that grasp something like ratings from newspaper and/orpassers-by have also been disclosed as in JP-A-4-307657 andJP-A-2007-18218 (Patent Documents 1 and 2).

SUMMARY OF THE INVENTION

However, the past ratings are measured in a state where a televisionbroadcasting program is being selected and displayed by a televisionreceiver and have nothing to do with whether viewers are actuallywatching the program or not in this case. In other words, “whether theprogram is being displayed or not” is only reflected on the ratings. Inreality, television receivers are on, but viewers may not view them inmany cases under the state “the program is being displayed”.

Because of the wide spread of recorders, an increased number of peoplerecord broadcasting programs and view them later rather than real-timeviewing.

For those reasons, it is difficult to say that the ratings calculated bythe past method typically and properly reflect the real state of viewingof television programs. Then, it is difficult for advertisers to fullyrecognize the effects of advertisements, demanding improvements.

Accordingly, it is desirable to create viewing information such as thenumber of viewers and the ratings in touch with reality.

According to an embodiment of the present invention, there is providedan imaged image data processing apparatus including obtaining means forobtaining imaged image data, which is imaged by an imaging section thatimages the sight of a user, extracting means for extractingidentification data for identifying a viewed program from the imagedimage data obtained by the obtaining means, and transmitting means fortransmitting the identification data extracted by the extracting meansto an external viewing information creating apparatus.

For example, the obtaining means may have the imaging section integrallyand obtain the imaged image data by imaging in the imaging section. Theimaging section may have a CCD sensor or a CMOS sensor as an imagingdevice.

The obtaining means may obtain the imaged image data by imaging in theimaging section through communication with the imaging section, which isprovided separately.

The extracting means may identify a displayed image on an image displayapparatus within the imaged image data by image analysis processing andextract the identification data from the displayed image on the imagedisplay apparatus if the displayed image of the image display apparatusexists within the imaged image data.

The extracting means may extract, as the identification data, all or apart of image data of the displayed image part detected within theimaged image data. Alternatively, the extracting means may extract, asthe identification data, text, a mark or a code within the displayedimage detected within the imaged image data.

The transmitting means may transmit the identification data extracted bythe extracting means to an external viewing information creatingapparatus over network communication.

The imaged image data processing apparatus may further include receivingmeans for receiving viewing information transmitted from an externalviewing information creating apparatus and display means for displayingand outputting the viewing information received by the receiving means.

According to another embodiment of the invention, there is provided aviewing information creating apparatus including receiving means forreceiving identification data for identifying a viewed program, which istransmitted from an external imaged image data processing apparatus,program identification processing means for performing processing ofidentifying a viewed program from the identification data received bythe receiving means, and information creating means for performingprocessing of creating viewing information based on the processingresult by the program identification processing means.

The viewing information creating apparatus may further includetransmitting means for transmitting the viewing information created bythe information creating means to an external machine.

According to another embodiment of the invention, there is provided aviewing information creating system including the imaged image dataprocessing apparatus and the viewing information creating apparatus.

According to another embodiment of the invention, there is provided animaged image data processing method including the steps of obtainingimaged image data, which is imaged by an imaging section that images thesight of a user, extracting identification data for identifying a viewedprogram from the obtained imaged image data, and transmitting theextracted identification data to an external viewing informationcreating apparatus.

According to another embodiment of the invention, there is provided aviewing information creating method including the steps of receivingidentification data for identifying a viewed program, which istransmitted from an external imaged image data processing apparatus,performing processing of identifying a viewed program from the receivedidentification data, and performing processing of creating viewinginformation based on the processing result by the processing ofidentifying a viewed program.

According to another embodiment of the invention, there is provided aviewing information creating apparatus including receiving means forreceiving imaged image data, which is transmitted from an externalimaged image data processing apparatus, extracting means for extractingidentification data for identifying a viewed program from the imagedimage data received by the receiving means, program identificationprocessing means for performing processing of identifying a viewedprogram from the identification data extracted by the extracting means,and information creating means for performing processing of creatingviewing information based on the processing result by the programidentification processing means.

According to another embodiment of the invention, there is provided aviewing information creating system including the imaged image dataprocessing apparatus and the viewing information creating apparatus. Inthis case, the imaged image data processing apparatus has transmittingmeans for transmitting imaged image data, which is imaged by an imagingsection that images the sight of a user, to the viewing informationcreating apparatus.

According to another embodiment of the invention, there is provided aviewing information creating method including the steps of receivingimaged image data, which is transmitted from an external imaged imagedata processing apparatus, extracting identification data foridentifying a viewed program from the received imaged image data,performing processing of identifying a viewed program from the extractedidentification data, and performing processing of creating viewinginformation based on the processing result of the processing ofidentifying a viewed program.

In other words, according to the embodiments of the invention, an imagedimage data processing apparatus side obtains imaged image data of thesight of a user, which is imaged by an imaging section worn by the user,for example.

Then, the imaged image data processing apparatus detects whether theimaged image data contains a displayed image which may be a broadcastedimage or not. In other words, whether an image of a broadcastingprogram, which is displayed on an image display of a televisionreceiver, for example, is contained therein or not is determined. Sincethe imaged image data is an imaged image of the sight of a user, it maybe determined that the user is actually watching the displayed imagedisplayed by a machine such as a television receiver if the displayedimage is contained in the imaged image data. For example, if thedisplayed image is a broadcast image, it may be determined that a useris actually viewing the broadcasting. If a user is not watching atelevision receiver by looking away or doing something different, forexample, the image of the broadcast program is no longer contained inthe imaged image data.

Accordingly, if the imaged image data contains a displayed image such asa broadcasting program, for example, which is displayed on a televisionreceiver, for example, identification data for identifying thebroadcasting program is extracted. For example, all or a part of thedisplayed image or text, a mark or a code may be extracted. Then, theidentification data is transmitted to a viewing information creatingapparatus.

The viewing information creating apparatus side identifies thebroadcasting program by image comparison or mark or code analysis basedon the received identification data and creates viewing informationbased on the identification result. In other words, the state that aviewer (user) is actually “watching” a television broadcasting programis detected and is used for creating viewing information.

The viewing information may be information on the viewing state forbroadcasting, such as the ratings (that is, the ratings for programs,time periods or commercials, the average ratings and/or instantratings), information on the number of viewers, the constituent ratio(on ages or genders) of viewers on a program, the ranking of viewedprograms and the ratio of the state that a television receiver is on anda program is actually being watched.

The imaged image data processing apparatus side may transmit imagedimage data, which is imaged by the imaging section worn by a user, forexample, directly to the viewing information creating apparatus, and theviewing information creating apparatus side may extract theidentification data.

The embodiments of the invention have an effect that the number ofviewers and/or the ratings in touch with reality can be grasped sincethe state that viewers are actually watching a television broadcastprogram is detected and is used for creating viewing information.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B are explanatory diagrams of an appearance example of animaging apparatus according to an embodiment of the invention;

FIG. 2 is an explanatory diagram of a system configuration exampleaccording to an embodiment of the invention;

FIG. 3 is a block diagram of an imaging apparatus according to anembodiment of the invention;

FIG. 4 is a block diagram of a server apparatus according to anembodiment of the invention;

FIG. 5 is a flowchart of System Operation Example I according to anembodiment of the invention;

FIGS. 6A to 6C are explanatory diagrams of the extraction ofidentification data according to an embodiment of the invention;

FIGS. 7A to 7D are explanatory diagrams of the extraction ofidentification data according to an embodiment of the invention;

FIG. 8 is a flowchart of System Operation Example II according to anembodiment of the invention;

FIG. 9 is a block diagram of an imaging apparatus corresponding toSystem. Operation Example III according to an embodiment of theinvention;

FIG. 10 is a block diagram of a server apparatus corresponding to SystemOperation Example III according to an embodiment of the invention;

FIG. 11 is a flowchart of System Operation Example III according to anembodiment of the invention; and

FIG. 12 is an explanatory diagram of another configuration example of animaged image data processing apparatus according to an embodiment of theinvention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the imaged image data processing apparatus, viewinginformation creating apparatus, viewing information creating system,imaged image data processing method and viewing information creatingmethod according to embodiments of the invention will be describedbelow. An imaging apparatus will be described herein as an example ofthe imaged image data processing apparatus. The imaged image dataprocessing apparatus according to an embodiment of the invention can beimplemented by a machine excluding an imaging apparatus, which will bedescribed later as variation examples.

A server apparatus will be described as an example of the viewinginformation creating apparatus.

A system that creates information on the ratings and the number ofviewers as viewing information will be described as an example.

Embodiments of the invention will be described in the following order:

[1. Appearance Example of Imaging Apparatus]; [2. System Configuration,Imaging Apparatus and Server Apparatus]; [3. System Operation ExampleI]; [4. System Operation Example II]; [5. System Operation Example III];and [6. Effects of Embodiments, Variations and Extension Examples] 1.Appearance Example of Imaging Apparatus

FIGS. 1A and 1B illustrate appearance examples of an imaging apparatus 1according to an embodiment of the invention.

FIG. 1A shows the imaging apparatus 1 functioning as an eyeglass-shapeddisplay camera. The imaging apparatus 1 has an attachment unit in aframe structure covering the half circumference of the head from bothsides to the back, for example, and is worn by a user over both ears, asshown in FIGS. 1A and 1B.

The imaging apparatus 1 includes an imaging lens 3 a to the front when auser is wearing it so as to image by handling the direction of the sightof the user as a subject direction.

The imaging apparatus 1 further includes a pair of display panels 2 aand 2 a for the right and left eyes, which are placed immediately infront of both eyes of a user, that is, where lenses of normal eyeglassesposition. The display panel section 2 a may be a liquid crystal panel,for example, and the transparent or translucent state can be obtained asshown in FIGS. 1A and 1B by controlling the transmittance. Thetransparent or translucent state of the display panel section 2 a allowsa user wearing it at all times to have everyday life without anytrouble.

A pair of the display panel sections 2 a may be provided correspondinglyfor both eyes or one display panel section 2 a may be provided for oneeye.

FIG. 1B shows the imaging apparatus 1 also to be attached to the head ofa user, without the display panel sections 2 a as shown in FIG. 1A. Forexample, the imaging apparatus 1 may be attached to the head of a userthrough an attachment unit to be hung on the ear. Under this state, theimaging lens 3 a is placed to the front so as to image by handling thedirection of the sight of a user as the subject direction.

Showing the imaging apparatus 1 to be attached to the head of a userthrough an eyeglass-shaped or head-attachable attachment unit in FIGS.1A and 1B, various structures may be considered for wearing the imagingapparatus 1 by a user. For example, a user may wear the imagingapparatus 1 through any type of attachment unit such as those of aheadphone type, neckband type and ear-hook type. Furthermore, a user maywear the imaging apparatus 1 by attaching it to normal eyeglasses, avisor or a headphone, for example, through an attachment unit such as aclip. Alternatively, it is not typically important to attach the imagingapparatus 1 to the head of a user.

The imaging apparatus 1 according to an embodiment of the invention maybe an apparatus to be attached to a user that full-time-images the sightin the direction of the field of view of a user, for example.

The full-time-imaging refers to imaging moving pictures at all timeswhile a user is wearing the imaging apparatus, and the full-time imagingaccording to this embodiment may also include an operation that imagesstill pictures at intervals of one, several or several-tens seconds. Inother words, the full-time imaging refers to continuous or intermittentimaging even without particular imaging operations performed by a user.

Notably, the imaging apparatus 1 that images still pictures and/ormoving pictures may have other forms than those shown in FIGS. 1A and1B. For example, the imaging apparatus 1 may have a microphone thatcollects external audio to obtain audio data in addition to image datain an imaging operation. Alternatively, the imaging apparatus 1 may havea speaker or earphone section for audio output.

A light emitting section such as an LED (light emitting diode) thatilluminates the subject direction or a flash light emitting section forimaging still pictures may be provided near the imaging lens 3 a.

2. System Configuration, Imaging Apparatus and Server Apparatus

FIG. 2 shows a system configuration example according to an embodimentof the invention. The system according to an embodiment of the inventionincludes multiple imaging apparatus 1 and a server apparatus 70, whichare connected over a network 60 such as the Internet for allowing datacommunication, as shown in FIG. 2.

The server apparatus 70 may be a server apparatus 70 of a rating companyor a server apparatus 70 of a broadcast station. For example, the serverapparatus 70 of a rating company may create viewing information for eachtelevision broadcast station. The server apparatus 70 of a broadcaststation may create viewing information of the broadcast station.Notably, the server apparatus 70 of one broadcast station may beconfigured to create viewing information for broadcast stations.

The network 60 may be the Internet, a cellular phone communicationnetwork, a PHS communication network, an ad-hoc network, a LAN or thelike.

As the system operation example will be described later, each of theimaging apparatus 1 extracts identification data for identifying aviewed program from imaged image data of the sight of a user. Theimaging apparatus 1 transmits the extracted identification data to theserver apparatus 70 over the network 60.

The server apparatus 70 receives the identification data transmittedfrom the imaging apparatus 1. Then, the server apparatus 70 performsprocessing of identifying a viewed program from the receivedidentification data and performs processing of creating viewinginformation based on the processing result.

A terminal apparatus, which is a machine like a personal computer, maybe connected to the network 60. For example, a terminal (advertiser'sterminal 80), which is placed in an office of an advertiser who providesa program and/or a terminal (broadcast station's terminal 81) placed ina broadcast station may be allowed to communicate with the serverapparatus 70 over the network 60. The server apparatus 70 may transmitthe created viewing information to the advertiser's terminal 80 orbroadcast station's terminal 81 so as to notify the ratings and/or thenumber of viewers of a program. The server apparatus 70 may alsotransmit the created viewing information to the imaging apparatus 1.

A configuration example of the imaging apparatus 1 will be describedwith reference to FIG. 3.

A system controller 10 may be a microcomputer including a CPU (centralprocessing unit), a ROM (Read Only memory), a RAM (Random AccessMemory), a non-volatile memory section and an interface section andfunctions as a control section that controls the entire imagingapparatus 1. The system controller 10 performs operational processingand exchanges control signals with the components through a bus 13 basedon a program held in the internal ROM to cause the components to performoperations.

An imaging section 3 has an imaging optical system, an imaging devicesection and an imaging signal processing section.

The imaging optical system in the imaging section 3 includes a lenssystem having the imaging lens 3 a shown in FIGS. 1A and 1B, anaperture, a zoom lens and a focus lens and a driving system for causingthe lens system to perform a focus operation or a zoom operation.

The imaging device section in the imaging section 3 includes asolid-state imaging device array that detects imaging light obtained bythe imaging optical system and generates an imaging signal by performingphotoelectric conversion. The solid-state imaging device array may be aCCD (Charge Coupled Device) sensor array or a CMOS (Complementary MetalOxide Semiconductor) sensor array.

The imaging signal processing section in the imaging section 3 mayinclude a sample hold/AGC (Automatic Gain Control) circuit, whichperforms gain adjustment or waveform adjustment on signals obtained bythe solid-state imaging device, and/or a video A/D converter and obtainsimaged image data, which is digital data. The imaging signal processingsection may also perform white balance processing, intensity processing,color signal processing and shake correction processing on imaged imagedata.

The imaging section 3 including the imaging optical system, imagingdevice section and imaging signal processing section performs imaging,whereby imaged image data is obtained.

The image data obtained by the imaging operation in the imaging section3 is processed by an imaging control section 6. The imaging controlsection 6 performs processing of converting imaged image data to thedata in a predetermined image data format and processing of supplyingthe converted imaged image data to a data extracting section 12, astorage section 14, a communication section 15 and/or a display controlsection 7 according to the operational situation, under the control ofthe system controller 10.

Based on an instruction by the system controller 10, the imaging controlsection 6 may control the ON/OFF of the imaging operation in the imagingsection 3, control the driving of the zoom lens and/or focus lens of theimaging optical system, control the sensitivity and/or frame rate of theimaging device section, control the parameter for processing in theimaging signal processing section and/or define processing to beperformed.

The data extracting section 12 may perform image analysis processing onthe imaged image data obtained by the imaging section 3 or the imagingcontrol section 6 and creates identification data.

The image analysis processing in this case refers to processing ofdetecting the displayed image on a display apparatus such as atelevision receiver in order to extract identification data.

The identification data refers to information to be used by the serverapparatus 70 for identifying a viewed program and is all or a part ofimage data of the broadcasted image part detected within imaged imagedata. Alternatively, the identification data may be information in text,a mark or a code within a displayed image detected within imaged imagedata. As operations by the data extracting section 12 will be describedlater, the identification data may be specifically those shown in FIGS.6B and 6C and 7B and 7D.

FIG. 6A shows an image functioning as imaged image data, which containsa television receiver, and the television receiver displays a broadcastprogram. FIGS. 6B and 6C are all or a part of the image of the broadcastprogram. This kind of image data may be used as the identification data.

FIG. 7A shows a case where a mark 100 such as a logo of a broadcaststation or text of the name of a broadcast station is displayed on thescreen of a broadcast program. FIG. 7B shows an example in which theimage of the mark 100 is used as the identification data in that case.

FIG. 7C shows a case where a code 101 like a two-dimensional barcodesuch as a QR code containing identification information such as the nameof a broadcast station is displayed on the screen of a broadcastprogram. FIG. 7D shows an example in which the image of the code 101 isused as the identification data.

The extraction of the identification data will be described later.

Notably, it is difficult to determine whether the display apparatuscontained in the imaged image data is displaying television broadcastingor not. For that reason, the displayed image on the display apparatuscontained in the imaged image data may not be a television broadcastimage, but it is not important to determine it for the extraction of theidentification data. This is because it can be identified by the serverapparatus 70 side.

The data extracting section 12 may be a microcomputer or a DSP (DigitalSignal Processor). Showing in the separate block from the systemcontroller 10 being a microcomputer in FIG. 3, the operations by thedata extracting section 12 may be implemented by software to be up andrunning in the system controller 10.

The configuration that provides display to a user in the imagingapparatus 1 may include a display section 2 and a display controlsection 7.

The display section 2 includes the display panel section 2 a, which maybe a liquid crystal panel, and a display driving section that drives thedisplay of the display panel section 2 a.

The display driving section includes a pixel driving circuit thatdisplays image data supplied from the imaging control section 6 on thedisplay panel section 2 a, which may be a liquid crystal display. Thepixel driving circuit applies a drive signal based on a video signal ata predetermined horizontal/vertical driving time to each of the pixelslaid out in a matrix form on the display panel section 2 a to implementthe display.

Under the control of the system controller 10, the display controlsection 7 drives the pixel driving circuit in the display section 2 tocause the display panel section 2 a to implement predetermined display.

In other words, the display panel section 2 a is caused to implement thedisplay by the imaging monitor in the imaging section 3, the playback ofimaged image data captured in the storage section 14, the display ofdata received by the communication section 15, display of a characterand so on.

For the display, the display control section 7 may perform intensitylevel adjustment, color correction, contrast adjustment, sharpness (edgeenhancement) adjustment and so on. The display control section 7 maycreate an enlarged image in which a part of image data is enlarged or areduced image, perform soft-focusing, pixelization, negative processing,highlighting of a part of an image, image effect processing such aschanging the entire color atmosphere, separate or synthesize images fordivision display of an imaged image, perform processing of creating acharacter image or a conceptual image and synthesizing the created imageto an imaged image.

The display control section 7 can control the display driving sectionbased on an instruction by the system controller 10 and thus controlsthe transmittance of the pixels of the display panel section 2 a toobtain a transparent or translucent state.

The storage section 14 is used for storage of data. The storage section14 may be used for storage of imaged image data, for example.

The storage section 14 may be a solid-state memory such as a RAM and aflash memory or may be an HDD (Hard Disk Drive).

The storage section 14 may be, instead of an internal recording medium,a portable recording medium such as a memory card internally containinga solid-state memory or a read write drive supporting a recording mediumsuch as an optical disk, a magnetooptical disk and a hologram memory.

Apparently, the storage section 14 may include both of an internallycontained memory such as a solid-state memory and an HDD and a readwrite drive for a portable recording medium.

The storage section 14 records and stores imaged image data under thecontrol of the system controller 10. The storage section 14 reads outrecorded data and supplies it to the system controller 10 and/or displaycontrol section 7 under the control of the system controller 10.

The communication section 15 performs the exchange of data with anexternal machine such as the server apparatus 70. The communicationsection 15 performs network communication through near fieldcommunication to a network access point over a wireless LAN orBluetooth, for example.

Next, FIG. 4 shows a configuration example of the server apparatus 70.

As described above, the server apparatus 70 is an apparatus thatreceives identification data from the imaging apparatus 1 throughcommunication over the network 60 and creates viewing information byusing the identification data.

The server apparatus 70 includes a server control section 72, a networkstorage section 71, a communication section 73, a program identificationprocessing section 74, a broadcast image obtaining section 75, a ratingcalculating section 76 and a broadcast image database 77.

The network storage section 71 may be implemented by an HDD, forexample, and is used for storing data such as identification datatransmitted from the imaging apparatus 1 over the network 60 ortemporarily storing data to be transmitted to the imaging apparatus 1,the advertiser's terminal 80 and/or the broadcast station's terminal 81,for example.

The communication section 73 performs data communication with thecommunication section 26 of the imaging apparatus 1 and/or thecommunication section of the advertiser's terminal 80 and the broadcaststation's terminal 81 over the network 60.

The server control section 72 performs important operational control asthe server apparatus 70. In particular, the server control section 72performs control operations for creating and/or transmitting viewinginformation such as the ratings.

The program identification processing section 74 performs processing ofidentifying a broadcast program based on identification data.

The broadcast image obtaining section 75 obtains a television broadcastimage (realtime broadcast image) currently on the air in each broadcaststation, for example. In a case where the broadcast image obtainingsection 75 is in the server apparatus 70 of a rating company, forexample, the broadcast image obtaining section 75 may be configured toreceive broadcast images of multiple broadcast stations at all times asa tuner apparatus for television broadcasting. If in the serverapparatus 70 of a broadcast station, the broadcast image obtainingsection 75 may be a device to which a broadcast image on the air by thebroadcast station can be input.

In a case where the identification data transmitted from the imagingapparatus 1 is image data of a broadcast image, obtaining the televisionbroadcast image currently on the air by the broadcast image obtainingsection 75 allows the program identification processing section 74 tocompare the image of the identification data and the broadcast image anddetermine the broadcast station that has broadcasted the image data,which is the identification data.

In other words, it is only important to configure the broadcast imageobtaining section 75 to obtain a broadcast image to be compared with theimage of the identification data and more specifically to obtain arealtime broadcast image of a broadcast station that needs to check theratings through the server apparatus 70.

In a case where the server apparatus 70 of a rating company needs toresearch the ratings of terrestrial broadcast stations and satellitebroadcast stations, for example, the broadcast image obtaining section75 may be configured to supply all of broadcast images of theterrestrial broadcast stations and satellite broadcast stations to theprogram identification processing section 74 at all times.

The program identification processing section 74 in this case canidentify the broadcast station which has broadcasted the image datafunctioning as identification data by comparing the image of theidentification data and each broadcast image. Alternatively, if theimage as the identification data does not agree with broadcast images ofany broadcast stations, it can be guessed that the image as theidentification data, which has been extracted by the imaging apparatus 1side, may not be a broadcast image (that is, not a broadcast image onthe air of broadcast stations subject to the ratings measurement atleast).

In a case where the server apparatus 70 of a broadcast station needs tocheck the ratings of the station, for example, the broadcast imageobtaining section 75 is configured to supply the current broadcast imageof the station to the program identification processing section 74 atall times.

The program identification processing section 74 in this case canidentify whether the image data as identification data is the broadcastimage of the station or not by comparing the image as the identificationdata and the broadcast image. If the image as the identification datadoes not agree with the broadcast image of the station, it can beguessed that the image as the identification data, which has beenextracted by the imaging apparatus 1 side, is not the broadcast image ofthe station (that is, not the current broadcast image at least).

The program identification processing section 74 may properly storerealtime broadcast images for a predetermined period of time temporarilyin consideration of the time lag for communication and/or processing inthe operation of comparing the image as identification data and therealtime broadcast image and then compare the realtime broadcast imagesin a period of time to some extent from the present to the past and theimage as identification data.

The broadcast image database 77 accumulates and stores past broadcastimages of one or multiple broadcast stations to be searched for viewinginformation such as the ratings. For example, the broadcast imagedatabase 77 may store past broadcast images for a specific period oftime such as past three days, past one week and past one month.

This becomes important for reflecting the fact that a user of theimaging apparatus 1 watches a recorded broadcast image to viewinginformation. In other words, if the image as identification data doesnot agree with a realtime broadcast image from the broadcast imageobtaining section 75, the image as identification data may be possiblyone in a recorded program being watched by a user.

In order to also reflect the viewing of a recorded program to viewinginformation, the program identification processing section 74 comparesthe image as identification data with a past broadcast image stored inthe broadcast image database 77 after the image as identification datadoes not agree with a realtime broadcast image. If the data asidentification data agrees with one image, it can be determined that theuser of the imaging apparatus 1 is watching past broadcasting by onebroadcast station.

In this way, the program identification processing section 74 canidentify the broadcast station being watched by a user by comparing theimage as identification data and a realtime broadcast image and/or apast broadcast image.

However, other identification methods are available instead of the imagecomparison. In a case where a logo, a mark, a code or text, for example,is transmitted as the identification data as described above, thebroadcast station being watched by a user may be identified by comparingthe image of the mark, for example, with the marks of broadcaststations, identifying the letters of the name of the broadcast stationor decoding the code such as a QR code, for example.

If the broadcast station relating to the identification data isidentified, the program identification processing section 74 notifiesthe broadcast station identification information to the server apparatus70. Alternatively, other information like a realtime broadcast image ora past broadcast image may be notified.

The ratings calculating section 76 counts the number of viewers and/orcalculates the ratings for each broadcast station to be searched basedon an instruction by the server control section 72.

The ratings can be calculated by using the number of viewers for eachbroadcast station based on the identification by the programidentification processing section 74 where the total number of imagingapparatus 1 that can communicate with the server apparatus 70 or thetotal number of imaging apparatus 1 during communication is thepopulation.

The ratings to be calculated may be the instant ratings, the ratings foreach program, the ratings for each commercial, the average ratings foreach time period, the ratings for realtime/recorded programs, programratings in consideration of the number of viewers of recorded programsand/or the like.

The viewing information calculated by the ratings calculating section76, that is, the number of viewers of each broadcast station or theratings as described above may be stored in the network storage section71 and/or can be transmitted through the communication section 73, underthe control of the server control section 72. In other words, theviewing information can be provided to the imaging apparatus 1,advertiser's terminal 80 and/or broadcast station's terminal 81.

The system configuration example and configurations of the imagingapparatus 1 and server apparatus 70 have been described for theillustration purpose only. Apparently, the addition or deletion of acomponent or components is possible according to the operation exampleor function to be actually implemented.

3. System Operation Example I

A system operation example to be implemented between the imagingapparatus 1 and server apparatus 70 as described above will bedescribed. FIG. 5 shows a processing operation by the imaging apparatus1 and a processing operation by the server apparatus 70. The processingin steps F101 to F106 in the imaging apparatus 1 is performed by theapplicable component or components under the control of the systemcontroller 10. FIG. 5 only shows the processing relating toidentification data to be used for creation of viewing information.

The processing in steps F201 to F205 in the server apparatus 70 isperformed by the applicable component or components under the control ofthe server control section 72. FIG. 5 only shows the processing relatingto the creation of viewing information.

In the imaging apparatus 1, the processing in steps F102 to F105 isrepeatedly performed in response to the start of imaging. In otherwords, the processing in steps F102 to F105 is repeated until theimaging in step F106 ends.

The imaging apparatus 1 performs the full-time imaging (moving pictureserial imaging or still picture imaging at predetermined time intervals)as described above. A user may wear the imaging apparatus 1 as shown inFIG. 1A or 1B, for example, and the system controller 10 advances theprocessing from step F101 to F102 when the power is on.

In step F102, the system controller 10 controls the data extractingsection 12 to perform analysis processing on imaged image data.

The data extracting section 12 performs the analysis processing on theimaged image data supplied from the imaging control section 6 fordetecting a display apparatus such as a television receiver.

The imaged image data obtained by the imaging section 3 or the imagingcontrol section 6 is an image of the sight of a user. When a user iswatching some display apparatus (such as a television receiver, amonitor of a personal computer, a screen of a cellular phone andothers), the display apparatus is contained in the imaged image data.

FIG. 6A shows the imaged image in a case where a user is watching atelevision receivers (that is, a television broadcast program) indoors,and the television receiver is imaged as a part of the indoor sight.

An area having a square contour is detected first in order to detect thescreen area of the display apparatus such as a television receiverwithin imaged image data. In particular, an area in a square at anaspect ratio 4:3 or 16:9 may be detected.

However, the square in this case includes not only a rectangle but alsoa form such as a trapezoid and a parallelogram. This is because thescreen of the imaged television receiver may not be a rectangular areatypically at some positions, angles of elevation and angles of a useragainst the television receiver. The part having a square (rectangular,trapezoidal or parallelogram) contour within an imaged image istherefore extracted on the imaged image data.

Notably, the screen in a rectangular form is generally shown in atrapezoidal or parallelogram form on the image according to thedirection of imaging, angle of elevation and/or position in imagingand/or the laws of perspective. For that reason, the analysis processingis also effective that guesses the direction of imaging, the angle ofelevation and so on based on the states of entire contours within animage (such as contours of the cabinet of the television receiver andsurrounding furniture) and extracting the trapezoid or parallelogram,which can be guessed as being originally (or really) rectangular.

After a square area is detected, whether the intensity within thedetected square area is high or not or whether the intensity changessignificantly among frames or not is determined.

If the inside of the square area has a high intensity or significantintensity changes, it can be guessed that the part may be some displayapparatus that displays something.

In other words, it can be guessed that a user is possibly watching atelevision broadcast program.

Notably, it is difficult to determine whether the one being actuallywatched by a user is a broadcasting program or not in this stage, but itis not a problem.

For example, a display apparatus may display various images such as aplayback image of a television broadcasting program or a recordedtelevision broadcasting program, a playback image on a medium such as aDVD, a playback image of a home video, an image on a monitor of apersonal computer, for example, and a photograph image imaged by adigital still camera.

It is not typically important for the analysis processing by the dataextracting section 12 to distinguish the broadcast images as describedabove and other images from the viewpoint of the creation of viewinginformation though the image of a television broadcasting programcurrently on the air or a playback image of a recorded televisionbroadcasting program is subject to the calculation of the ratings. Thisis because the server apparatus 70 side can also determine whether thesubject image is an image of a broadcast program or not.

After detecting the image of the display apparatus currently displayingwithin imaged image data, the data extracting section 12 moves from stepF103 to F104 to extract identification data from the data of thedisplayed image on the display apparatus.

On the imaged image in FIG. 6A, for example, identification data isextracted from the imaged image since the image of the display apparatuscurrently displaying is detected as a square area. As the identificationdata, image data of the image itself within the square area may beextracted, as shown in FIG. 6B, for example. In other words, theidentification data may be the image being viewed by a user on a displayapparatus.

Alternatively, the identification data may be a part of the image withinthe square area, as shown in FIG. 6C.

Alternatively, after an area having a square contour is detected,whether text is contained within the square or not may be determined.For example, pattern recognition may be performed on the text. Notably,whether an image, which can be guessed as a logo or a mark, is containedtherein or not may be detected by storing logos and/or marks in advanceand performing image pattern matching therewith.

Then, if the mark 100 such as a logo mark and text is displayed within asquare area as shown in FIG. 7A, for example, the image of the mark 100such as a logo mark and text may be extracted as shown in FIG. 7B, whichis then handled as the identification data. If the mark 100 is text,character recognition may be performed thereon, and the resulting textdata may be handled as the identification data.

If the code 101 such as a QR code is displayed within the square area asshown in FIG. 7C, the image of the code 101 may be extracted and behandled as the identification data, as shown in FIG. 7D. A decodingprocessing function if any for the code 101 in the data extractingsection 12 may perform decoding processing thereon, and the resultingdecoded data may be handled as the identification data.

After the data extracting section 12 extracts the identification data,the system controller 10 advances the processing to step F105 totransmit the identification data through the communication section 15 tothe server apparatus 70.

If imaging is being continued, that is, if the power is not off, theprocessing returns from step F106 to F102.

Notably, if the image of the display apparatus currently displaying isnot detected within the imaged image data as a result of the analysis onthe imaged image data in step F102, the processing moves from step F103to F106 and returns to step F102 if the imaging is being continued. Forexample, the data extracting section 12 may notify the system controller10 of the fact that no display apparatus exist within the imaged imagedata. The system controller 10 in response thereto after step F102controls the next imaged image data to be transferred from the imagingcontrol section 6 to the data extracting section 12 and controls thedata extracting section 12 to perform the analysis processing on theimaged image data.

The server apparatus 70 performs the processing at and after step F202every time identification data is transmitted from the imaging apparatus1.

In other words, the server control section 72 receives identificationdata from one imaging apparatus 1 through the communication section 73and then advances the processing from step F201 to F202 to control theprogram identification processing section 74 to perform the programidentification processing using the received identification data.

If the identification data is a broadcast image data, the programidentification processing section 74 identifies the broadcast station bycomparing the identification data with a realtime broadcast image orpast broadcast images. If not, the processing may result in noapplicable broadcast station.

If the identification data is text, a mark or a code, the broadcaststation is identified by comparing between marks, determining theletters or decoding the code. If not, the processing may result in noapplicable broadcast station.

If the broadcast station is not identified, the program identificationprocessing section 74 notifies the server control section 72 of the factthat the broadcast station is not identified. In this case, the servercontrol section 72 returns the processing from step F203 to F201 andawaits the reception of the next identification data.

Though there may be a situation where a user on the imaging apparatus 1side is watching an image, which is not a television broadcast image, ona display apparatus, as described above, the image data extracted asidentification data in this case does not agree with broadcast images asa result of the comparison, and the processing results in no applicablebroadcast station.

If the broadcast station is identified, the program identificationprocessing section 74 notifies the server control section 72 of theinformation on the identified broadcast station. Notably, theinformation on the type of image between a realtime broadcast image anda past broadcast image may be also notified.

In this case, the server control section 72 advances the processing fromstep F203 to F204 and controls the ratings calculating section 76 toperform the processing for viewing information.

The ratings calculating section 76 may perform processing of counting upthe number of viewers for the broadcast station, which is currentlyidentified, calculating the ratings and/or updating them based on thecalculation result. The ratings may be the instant ratings, the ratingsfor each time period, the ratings for each program, the ratings for eachcommercial, the ratings for realtime/recorded programs or the programratings in consideration of the number of viewers of recorded programsand/or the like.

Then, the server control section 72 in step F205 performs processing ofnotifying the viewing information calculated (updated) by the ratingscalculating section 76. That is, the server control section 72 transmitsthe viewing information through the communication section 73 to theadvertiser's terminal 80 and/or broadcast station's terminal 81 andprovides the ratings research result.

The processing in step F205 may be performed at predetermined timeintervals or may be performed all at once a day instead of every timeafter step F204.

In the system operation example as described above, the processing basedon identification data from many imaging apparatus 1 is performed in theserver apparatus 70, and viewing information is created. Then, theviewing information is notified to advertisers and/or broadcaststations.

4. System Operation Example II

FIG. 8 shows System Operation Example II. Steps F101 to F105 by theimaging apparatus 1 in FIG. 8 are the same as steps F101 to F105 in FIG.5, and the processing in steps F201 to F204 by the server apparatus 70are also the same as the processing in steps F201 to F204 in FIG. 5.

In this example, the server apparatus 70 transmits the viewinginformation calculated at a predetermined time to the imaging apparatus1 in the ratings notification processing in step F205.

On the imaging apparatus 1 side after the processing in steps F102 toF105, the system controller 10 determines whether viewing informationhas been received from the server apparatus 70 in step F120. If viewinginformation has been received from the server apparatus 70 through thecommunication section 15, the system controller 10 advances theprocessing to step F121 to control the display section 2 to implementthe display based on the received viewing information.

For example, viewing information on the program being currently viewedby a user or the ratings of broadcast stations may be displayed.

By notifying the current ratings of broadcast stations by the serverapparatus 70, for example, the imaging apparatus 1 can display a list ofthe ratings of broadcast stations.

The imaging apparatus 1 side can display the ratings of the programbeing watched by a user by notifying the imaging apparatus 1, which isthe sender of identification data, of the ratings of the broadcaststation identified from the identification data by the server apparatus70.

Alternatively, by notifying the ratings for broadcast stations and theinformation on the name of the broadcast station identified based onidentification data to the imaging apparatus 1 by the server apparatus70, the imaging apparatus 1 side may not only display the ratings of theprogram currently being watched by a user but also display the ratingsof other stations.

In other words, according to System Operation Example II in FIG. 8, auser can learn the current ratings while viewing televisionbroadcasting.

5. System Operation Example III

Next, System Operation Example III will be described.

FIGS. 9 and 10 show configurations of the imaging apparatus 1 and serverapparatus 70 that implement System Operation Example III.

The imaging apparatus 1 in FIG. 9 excludes the data extracting section12 in the configuration illustrated in FIG. 3.

The server apparatus 70 in FIG. 10 includes a data extracting section 77in addition to the configuration in FIG. 4. The processing operation bythe data extracting section 77 is the same as that of the dataextracting section 12 on the imaging apparatus 1 side as describedabove.

FIG. 11 shows processing by the imaging apparatus 1 and the serverapparatus 70.

In this case, the imaging apparatus 1 after imaging is started in stepF301 performs processing of transmitting imaged image data in step F302until the end of imaging in step F303.

In other words, in an operation for moving picture imaging, the systemcontroller 10 controls to continuously perform the processing oftransmitting the imaged image data obtained by the imaging section 3 andimaging control section 6 to the server apparatus 70 through thecommunication section 15.

In an operation of imaging still pictures intermittently, the systemcontroller 10 controls to transmit the imaged image data to the serverapparatus 70 through the communication section 15 every time imaging isperformed.

The server apparatus 70 performs processing at and after step F401 everytime imaged image data is transmitted from the imaging apparatus 1.

In other words, the server control section 72 after receiving imagedimage data from one imaging apparatus 1 through the communicationsection 73 advances the processing from step F401 to F402 to control thedata extracting section 77 to perform analysis processing on thereceived imaged image data.

In other words, the data extracting section 77 performs analysisprocessing on the imaged image data for detecting a display apparatussuch as a television receiver. The processing is the same as theprocessing by the data extracting section 12 in step F102 shown in FIG.5 and determines whether a displayed image by a display apparatus iscontained in the imaged image data or not by determining whether theimaged image data internally contains an area having a square contourwith a high intensity or many intensity changes inside.

If any image of a display apparatus currently displaying is not detectedwithin the imaged image data as a result of the analysis on the imagedimage data in step F402, the server control section 72 returns theprocessing from step F403 to F401 and awaits the reception of the nextimaged image data.

On the other hand, if the data extracting section 77 determines that animage of a display apparatus exists within the imaged image data, theprocessing moves from step F403 to F404 where identification data isextracted.

In other words, like the processing by the data extracting section 12 instep F104 shown in FIG. 5, all or a part of broadcast image data withinthe square area is extracted as identification data. Alternatively,text, a mark or a code, for example, within the square area may beextracted as identification data.

Then, the server control section 72 in step F405 controls the programidentification processing section 74 to perform program identificationprocessing by using the identification data.

If the identification data is broadcast image data, the programidentification processing section 74 identifies the broadcast station bycomparing it with a realtime broadcast image or a past broadcast image.If not, the processing may result in no applicable broadcast station. Ifthe identification data is text, a mark or a code, the broadcast stationis identified by comparing between marks, determining the letters ordecoding the code. If, not, the processing may result in no applicablebroadcast station.

If the broadcast station is not identified, the program identificationprocessing section 74 notifies the server control section 72 of the factthat the broadcast station is not identified. In this case, the servercontrol section 72 returns the processing from step F406 to F401 andawaits the reception of the next imaged image data.

If the broadcast station is identified, the program identificationprocessing section 74 notifies the server control section 72 of theinformation on the identified broadcast station. Notably, theinformation on the type of the image between a realtime broadcast imageand a past broadcast image may be also notified.

In this case, the server control section 72 advances the processing fromstep F406 to F407 and controls the ratings calculating section 76 toperform the processing for viewing information.

The ratings calculating section 76 may perform processing of counting upthe number of viewers for the broadcast station, which is currentlyidentified, calculating the ratings (the instant ratings, the ratingsfor each time period, the ratings for each program, the ratings for eachcommercial, the ratings for realtime/recorded programs or the programratings in consideration of the number of viewers of recorded programsand/or the like) and/or updating them based on the calculation result.

Then, the server control section 72 in step F408 performs processing ofnotifying the viewing information calculated (updated) by the ratingscalculating section 76. That is, the server control section 72 transmitsthe viewing information through the communication section 73 to theadvertiser's terminal 80 and/or broadcast station's terminal 81 andprovides the ratings search result.

The processing in step F408 may be performed at predetermined timeintervals or may be performed all at once a day instead of every timeafter step F407. In step F408, viewing information may be transmitted tothe imaging apparatus 1, and the imaging apparatus 1 in response to thereception of the viewing information may display and output the viewinginformation.

In the system operation example as described above, the processing ofcreating viewing information based on imaged image data from manyimaging apparatus 1 is performed in the server apparatus 70, and viewinginformation is created. Then, the viewing information is notified toadvertisers and/or broadcast stations.

6. Effects of Embodiments, Variations and Extension Examples

The embodiments described above allow the grasp of the number of viewersand/or the ratings in touch with reality since the state that a vieweris actually watching a television broadcast program is detected and isused for creating viewing information.

If the imaged image data imaged by the imaging apparatus 1 contains animage displayed by a display apparatus such as a television receiver, itcan be determined that the displayed image is being watched by a user.

Therefore, it may be determined that a user is actually viewing thebroadcasting if the broadcast program (broadcast station) is identifiedby the displayed image itself or text, a mark or a code, for example,contained in the displayed image. This can exclude the situation where atelevision receiver is shown but nobody is watching it from themeasurement for the ratings. As a result, viewing information in touchwith reality can be obtained.

In addition, as the number of users using the imaging apparatus 1increases, the degree of effectiveness of the viewing information to beobtained increases.

Using a displayed image as the identification data allows thecalculation of the ratings in consideration of what is actually viewedby users for each program or commercial. For example, a case where auser goes to a different room while a commercial is being displayed, forexample, may be reflected on the viewing information by decreasing theratings for the time for the commercial. For example, greatly effectiveinformation can be provided to an advertiser, such as how much thecommercial thereby is being viewed.

The information in touch with reality including realtime viewing andrecorded viewing can also be obtained. Thus, programs with high and lowrealtime characteristics can be also found as viewing situations basedon actual situations.

The enjoyment that a user can view by being aware of the ratings can becreated by the display of viewing information on the imaging apparatus 1side as in System Operation Example II.

The imaging apparatus 1 in a simple configuration is applicable to thepresent system since the function of transmitting imaged image data tothe server apparatus 70 is only important (that is, creation ofidentification data is not necessary) in the imaging apparatus 1 side inthe case of System Operation Example III.

The invention is not limited to the embodiments above, but variousvariation examples and extension examples are considerable asconfiguration examples and/or processing examples of the imagingapparatus 1.

Having described the imaging apparatus 1 as an example of the imagedimage data processing apparatus according to the embodiment of theinvention, various apparatus are applicable as the imaged image dataprocessing apparatus instead of imaging apparatus.

For example, the imaged image data processing apparatus 20 as shown inFIG. 12 in which an imaging section is provided separately may beconsidered as an example of the imaged image data processing apparatusaccording to the embodiment of the invention.

An image obtaining section 21 is communicable with a wearable imagingapparatus as shown in FIGS. 1A and 1B by wired or wirelesscommunication.

The image obtaining section 21 receives imaged image data from theimaging apparatus and performs image analysis and extraction ofidentification data on the imaged image data in the data extractingsection 22. Then, the identification data transmitting section 23transmits the identification data to the server apparatus 70.

The imaged image data processing apparatus 20 without any imagingfunction in this way may be provided separately from a wearable imagingapparatus as shown in FIGS. 1A and 1B and is allowed to communicate withthe imaging apparatus and may be allowed to transmit identification datato the server apparatus 70.

Image data is extracted as identification data, for example, accordingto the embodiment described above, but audio data may be used asidentification data.

For example, a microphone may be provided in the imaging apparatus 1,and audio data obtained by the microphone, that is, audio data which canbe guessed as the audio of a broadcast program may be added toidentification data along with an image, a mark, a code or text, forexample.

In this case, the server apparatus 70 side can use the audio data tomatch with the tune on the air, title-calling, voice of a performerand/or audio of background music to identify a broadcast program.

The imaging apparatus 1 may transmit identification data and imagingtime information of imaged image data from which the identification datais extracted such that the server apparatus 70 side can performeffective image comparison by specifying the time.

Biological information of a user of the imaging apparatus 1 may betransmitted along with identification data.

For example, the imaging apparatus 1 may include a configurationfunctioning as a biological sensor and detect biological information ofa user. The biological information may be a pulse rate, a heartbeat, anelectrocardiogram information, an electromyogram, breathing information(such as the speed, depth and amount of ventilation of breathing),sweating, GSR (Galvanic skin reaction), blood pressure, arterial oxygensaturation, skin surface temperature, brain waves (such as informationon α, β, θ and δ waves), blood flow change, the state of the eye or thelike.

The biological sensor may be placed within the attachment frame of theeyeglass-shaped imaging apparatus 1 so as to be in contact with a sideor the back of the head of a user for detecting information as describedabove. Alternatively, the biological sensor may be provided separatelyfrom the attachment frame part of the imaging apparatus 1 and may beattached to a predetermined part of the body.

Providing such a biological sensor allows the estimation of the emotionof a user while the user is viewing television broadcasting. Forexample, it may be estimated that a user is “enjoying”, “excited” or“sad” or has “few emotional changes”, for example.

The estimated emotion information may be created, and the estimatedemotion information may be transmitted when identification data istransmitted to the server apparatus 70. Then, the server apparatus 70can obtain information on the emotion of a user viewing the identifiedprogram.

This means that the number of people who are actually happy by viewing ahappy program or the number of people who has a sad feeling by viewing asad scene in a program can be counted. In other words, the emotion ofviewers on the contents of a program can be determined, which may beeffective information in addition to the ratings.

According to the embodiment above, the server apparatus 70 obtains, asviewing information, the number of viewers calculated by the ratingscalculating section 76 (the estimated number of views or the estimatednumber of viewers) or the ratings (such as the instant ratings, theratings for each time period, the ratings for each program, the ratingsfor each commercial, the ratings for realtime/recorded programs and theprogram ratings in consideration of the number of viewers of recordedprograms).

According to the invention, not only the ratings and the number ofviewers as described above but also various kinds of viewing informationcan be obtained.

For example, information on the constituent ratio of viewers can beobtained. The constituent ratio may be the ratio of males to females oron ages of viewers watching a program, for example. For example, a userusing the imaging apparatus 1 may register his or her gender and/or ageas personal information in advance, and a server apparatus 2 may holdthe registered information in a user database along with the machine IDof the imaging apparatus 1. The imaging apparatus 1 may transmit themachine ID to the server apparatus 70 along with identification data instep F105 in FIG. 5 or 8 (or the imaged image signal in step F302 inFIG. 11). Thus, the server apparatus side can identify the attribute(gender and/or age) of the user based on the machine ID, which allowsthe counting of the number of viewers for each gender or age and thecreation of information on the constituent ratio on genders and/or agesof viewers.

Instead of the server registration of a user attribute in advance, theimaging apparatus 1 may store the attribute of a user and transmit theattribute information of the user when identification data and/or imagedimage data is transmitted, which also allows the creation of the sameinformation.

The ranking information of viewed programs can also be created as theviewing information. For example, the number of viewers for eachbroadcasting in a certain time period may be counted to create rankinginformation describing the broadcast station the program of which isviewed by viewers in the time period, without calculating the ratings.Apparently, the ranking information for commercials, the rankinginformation for programs and so on can be created instead of the one fortime periods.

Furthermore, as the viewing information, information can be createdwhich is the ratio of whether broadcasting is actually viewed or notwhen a television receiver is on. In a case where the server apparatus 2performs image analysis on imaged image data of the imaging apparatus 1as shown in FIG. 11, for example, it can be determined that a user isnot viewing broadcasting if no broadcast images are recognizedcontinuously to some extent even though broadcast images of a televisionreceiver are recognized within imaged images. In other words, thesituation can be determined in which a user is not watching broadcastingby doing something different while a television receiver is on. Based onthe analysis result, the ratio can be calculated which describes whethera target broadcast program or commercial is actually viewed or not.

According to the invention, various kinds of information on viewingsituations can be created as the viewing information in this way, forexample, and advantageous information can be provided to broadcaststations and/or advertisers.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

What is claimed is:
 1. A viewing information creating apparatuscomprising: at least one processor; at least one storage medium havingencoded thereon computer-executable instructions that, when executed bythe at least one processor, cause the at least one processor to: receiveimage data from at least one external imaging apparatus worn by a user,the image data comprising a sequence of images recorded over time by theat least one external imaging apparatus; analyze the image data receivedover time to identify a portion of at least one image, of the sequenceof images of the image data, that corresponds to a screen area of adisplay apparatus in operation; and in response to identifying a portionof at least one image that corresponds to a screen area of a displayapparatus in operation: analyze the portion of the at least one imagethat corresponds to the screen area to obtain, from the portion,identification data for identifying at least one program that wasdisplayed on the screen area of the display apparatus at a time the atleast one image was recorded; identify the at least one program based onthe identification data; and create viewing information based on theidentification of the at least one program.
 2. The viewing informationcreating apparatus of claim 1, wherein: the at least one externalimaging apparatus records the image data by acquiring images in adirection of a field of view of the user.
 3. The viewing informationcreating apparatus of claim 2, wherein: analyzing the image datareceived over time to identify a portion of at least one image, of thesequence of images of the image data, that corresponds to a screen areaof a display apparatus in operation comprises analyzing images of theimage data to determine whether the portion indicates featuresrepresentative of a display apparatus; and analyzing the portion of theat least one image to obtain the identification data in response toidentifying the portion comprises analyzing the portion in response toidentifying that the portion indicates features representative of adisplay apparatus.
 4. The viewing information creating apparatus ofclaim 1, wherein: analyzing the portion comprises comparing displayedprogram content indicated by the portion to a plurality of broadcastprograms to identify a broadcast station that has broadcasted thedisplayed program content; and identifying the at least one programcomprises identifying the at least one program based on the identifiedbroadcast station.
 5. The viewing information creating apparatus ofclaim 1, wherein: analyzing the portion comprises identifying, indisplayed program content indicated by the portion, at least oneindicator of the at least one program; and identifying the at least oneprogram comprises identifying the at least one program based on the atleast one indicator.
 6. The viewing information creating apparatus ofclaim 5, wherein: the at least one indicator comprises at least one of alogo, a mark, a code or text.
 7. The viewing information creatingapparatus of claim 1, wherein: the at least one external imagingapparatus comprises a plurality of external imaging apparatuses; and theviewing information comprises information on a number of users fromusers of the plurality of external imaging apparatuses that viewed theat least one program.
 8. The viewing information creating apparatus ofclaim 1, wherein: the computer-executable instructions, when executed bythe at least one processor, further cause the at least one processor to:receive biological information of the user; and create the viewinginformation based on the identification of the at least one program andthe detected biological information.
 9. The viewing information creatingapparatus of claim 8, wherein: the biological information comprisesinformation on a state of the eye of the user.
 10. The viewinginformation creating apparatus of claim 8, wherein: the biologicalinformation is acquired by at least one sensor monitoring at least onecondition of the user wearing the at least one external imagingapparatus.
 11. The viewing information creating apparatus of claim 8,wherein: the computer-executable instructions, when executed by the atleast one processor, further cause the at least one processor to:determine, based on the biological information of the user, emotioninformation of the user while the user is viewing the at least oneprogram.
 12. The viewing information creating apparatus of claim 11,wherein: the at least one external imaging apparatus comprises aplurality of external imaging apparatuses; and the viewing informationcomprises information on users of the plurality of external imagingapparatuses that viewed the at least one program and emotion informationof each of the users while the user viewed the at least one program. 13.The viewing information creating apparatus of claim 8, wherein: thecomputer-executable instructions, when executed by the at least oneprocessor, further cause the at least one processor to: transmit theviewing information to the external imaging apparatus.
 14. The viewinginformation creating apparatus of claim 8, wherein: thecomputer-executable instructions, when executed by the at least oneprocessor, further cause the at least one processor to: in response toidentifying that no portion of at least one first image of the sequenceof images corresponds to a screen area of a display apparatus inoperation, generating an indication that the at least one first imagedoes not correspond to a screen area of a display apparatus inoperation.
 15. A viewing information creating system comprising: animage data processing apparatus; and a viewing information creatingapparatus; wherein: the image data processing apparatus is configuredto: transmit acquired image data to the viewing information creatingapparatus, wherein the image data is recorded by at least one externalimaging apparatus worn by a user and comprises a sequence of imagesrecorded over time in a direction of a field of view of the user; andthe viewing information creating apparatus comprises: at least oneprocessor; at least one storage medium having encoded thereoncomputer-executable instructions that, when executed by the at least oneprocessor, cause the at least one processor to: receive the image datafrom the image data processing apparatus; analyze the image datareceived over time to identify a portion of at least one image, of thesequence of images of the image data, that corresponds to a screen areaof a display apparatus in operation; and in response to identifying aportion of at least one image that corresponds to a screen area of adisplay apparatus in operation: analyze the portion of the at least oneimage that corresponds to the screen area to obtain, from the portion,identification data for identifying at least one program that wasdisplayed on the screen area of the display apparatus at a time the atleast one image was recorded; identify the at least one program based onthe identification data; and create viewing information based on theidentification of the at least one program.
 16. The viewing informationcreating system of claim 15, wherein: the computer-executableinstructions, when executed by the at least one processor, further causethe at least one processor to generate rating information relating tothe at least one program; and the viewing information comprises therating information.
 17. A viewing information creating methodcomprising: with at least one processor: receiving image data from atleast one external imaging apparatus worn by a user, the image datacomprising a sequence of images recorded over time by the at least oneexternal imaging apparatus and comprises images in a direction of afield of view of the user; analyzing the image data received over timeto identify a portion of at least one image, of the sequence of imagesof the image data, that corresponds to a screen area of a displayapparatus in operation; and in response to identifying a portion of atleast one image that corresponds to a screen area of the displayapparatus in operation: analyzing the portion of the at least one imagethat corresponds to the screen area to obtain, from the portion,identification data for identifying at least one program that wasdisplayed on the screen area of the display apparatus at a time the atleast one image was recorded; identifying the at least one program basedon the identification data; and creating viewing information based onthe identification of the at least one program.
 18. A viewinginformation creating apparatus comprising: at least one processor; atleast one storage medium having encoded thereon computer-executableinstructions that, when executed by the at least one processor, causethe at least one processor to: receive image data from at least oneexternal imaging apparatus worn by a user, the image data comprisingdata a sequence of images recorded over time by the at least oneexternal imaging apparatus and comprising images in a direction of afield of view of the user; determine at least one program the useractually watched while wearing the at least one external imagingapparatus recording the image data, wherein the determining comprises:analyzing the image data received over time to identify a portion of atleast one image, of the sequence of images of the image data, thatcorresponds to a screen area of a display apparatus in operation; and inresponse to identifying a portion of at least one image that correspondsto a screen area of the display apparatus in operation: analyzing theportion of the at least one image that corresponds to the screen area toobtain, from the portion, identification data for identifying at leastone program that was displayed on the screen area of the displayapparatus at a time the at least one image was recorded; identifying theat least one program based on the identification data; determining,based on the identification data, a state of the user indicating whetherthe user has actually watched the at least one program; and creatingviewing information based on the identification of the at least oneprogram and the determined state of the user.
 19. The viewinginformation creating apparatus of claim 18, wherein: thecomputer-executable instructions, when executed by the at least oneprocessor, further cause the at least one processor to: in response toidentifying that no portion of at least one first image of the sequenceof images corresponds to a screen area of a display apparatus inoperation, generating an indication that the at least one first imagedoes not correspond to a screen area of a display apparatus in operation20. The viewing information creating apparatus of claim 18, wherein:analyzing the image data received over time to identify a portion of atleast one image, of the sequence of images of the image data, thatcorresponds to a screen area of a display apparatus in operationcomprises analyzing images of the image data to determine whether theportion indicates features representative of a display apparatus; andanalyzing the portion of the at least one image to obtain theidentification data in response to identifying the portion comprisesanalyzing the portion in response to identifying that the portionindicates features representative of a display apparatus.